Optical flow registration of panchromatic / multi-spectral image pairs

ABSTRACT

A method for processing remotely acquired imagery includes obtaining imagery data defining a first image of a panchromatic image type, the first image having a first spatial resolution, and obtaining imagery data defining a second image of a multi-spectral image type, the second image having a second spatial resolution lower than the first spatial resolution. The method also includes obtaining a mapping function specifying a position of pixels in the first image with respect to pixels in the second image and adapting the mapping function to a high common spatial resolution higher than the second spatial resolution. The method further includes generating a third set of imagery data defining a third image of a panchromatic type based on the first set of imagery data and the adapted mapping function and having the high common spatial resolution and adjusting the mapping function based on a first difference between the first and the third images at the high common spatial resolution.

BACKGROUND OF THE INVENTION

1. Statement of the Technical Field

The invention concerns image processing, and more particularly, an imageprocessing method for images having different spatial and spectralresolutions.

2. Description of the Related Art

In the field of remote image sensing, two common types of images includepanchromatic imagery and multi-spectral imagery. Panchromatic imagery isimagery that is obtained by a remote sensing device with a sensordesigned to detect electromagnetic energy in only one very broad band.This one very broad band typically includes most of the wavelengths ofvisible light. Panchromatic imagery has the advantage of offering veryhigh spatial resolution. In contrast, multi-spectral imagery istypically created from several narrow spectral bands within the visiblelight region and the near infrared region. Consequently, amulti-spectral image is generally comprised of two or more image datasets, each created by sensors responsive to different portions of theoptical spectrum (e.g., blue, green, red, infrared). Multi-spectralimages are advantageous because they contain spectral information whichis not available from a similar panchromatic image. However,multi-spectral images typically have a lower spatial resolution ascompared to panchromatic images.

It is often desirable to enhance a multi-spectral image with the highresolution of a panchromatic image and vice versa. Typically thisprocess is referred to as “fusing” of the image pair. In general, thereare several requirements for successfully accomplishing the fusingprocess. One requirement is the need for registration of the two images.The registration process involves a determination of where each pixel inthe panchromatic image maps to a location in the multi-spectral image.This process must generally be accomplished with great accuracy for bestresults. For example, it is desirable for each pixel in the pan image tobe mapped to the multi-spectral image with an accuracy of less than 0.1panchromatic pixel radius.

A number of conventional methods exist for registering image pairs. Inone method, registration is estimated based on the phase shift in theFourier domain. That is, one of the images is shifted in the Fourierdomain to match the second image. In another method, the sensor modelfor the acquired images is adjusted to project a small number of matchpoints to a common flat earth. Typically, in such methods, the imagedlocation is assumed to be static and nearly flat, resulting in a smoothmapping such as an affine or projective transform. In some instances,known topographical information, such as in a digital elevation model(DEM), may be used to adjust the transform. However, such mappingmethods can require an extensive amount of computation to match up andalign all the pixels in the image pair to be combined.

SUMMARY OF THE INVENTION

The present invention concerns systems and methods for processingremotely acquired imagery. In a first embodiment of the presentinvention, a method for processing remotely acquired imagery isprovided. The method includes the steps of obtaining imagery datadefining a first image of a panchromatic image type, the first imagehaving a first spatial resolution; obtaining imagery data defining asecond image of a multi-spectral image type, the second image having asecond spatial resolution lower than the first spatial resolution;obtaining a mapping function specifying a position of pixels in thefirst image with respect to pixels in the second image; adapting themapping function to a high common spatial resolution higher than thesecond spatial resolution; generating a third set of imagery datadefining a third image of a panchromatic type based on the first set ofimagery data and the adapted mapping function and having the high commonspatial resolution; and adjusting the mapping function based on a firstdifference between the first and the third images at the high commonspatial resolution.

In a second embodiment of the present invention, a system for processingremotely acquired imagery data is provided. The system includes a massstorage for receiving imagery data comprising imagery data defining afirst image of a panchromatic image type having a first spatialresolution and a second image of a multi-spectral image type, the secondimage having a second spatial resolution lower than the first spatialresolution. The system also includes a processing element configuredfor: obtaining a mapping function specifying a position of pixels in thefirst image with respect to pixels in the second image; adapting themapping function to a high common spatial resolution higher than thesecond spatial resolution; generating a third set of imagery datadefining a third image of a panchromatic type based on the first set ofimagery data and the adapted mapping function and having the secondspatial resolution; and adjusting the mapping function based on a firstdifference between the first and the third images at the high commonspatial resolution.

In a third embodiment of the present invention, a computer-readablestorage, having stored thereon a computer program for processingremotely acquired imagery is provided. The computer program includes aplurality of code sections executable by a computer to cause thecomputer to perform the steps of: obtaining imagery data defining afirst image of a panchromatic image type, the first image having a firstspatial resolution; obtaining imagery data defining a second image of amulti-spectral image type, the second image having a second spatialresolution lower than the first spatial resolution; obtaining a mappingfunction specifying a position of pixels in the first image with respectto pixels in the second image; adapting the mapping function to a highcommon spatial resolution higher than the second spatial resolution;generating a third set of imagery data defining a third image of apanchromatic type based on the first set of imagery data and the adaptedmapping function and having the high common spatial resolution; andadjusting the mapping function based on a first difference between thefirst and the third images at the high common spatial resolution.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a computer system within which a set ofinstructions operate according to an embodiment of the invention.

FIG. 2 is a flowchart of steps in an exemplary method for processingremotely acquired imagery according to an embodiment of the presentinvention.

FIG. 3 is a plot showing an example of a spectral response of sensorsused to create a panchromatic image and a multi-spectral image that isuseful for describing the present invention.

FIG. 4 is a conceptual illustration showing how spectral weights areused in a downsample processing of a multi-spectral image for decreasingthe spectral resolution of the multi-spectral image that is useful fordescribing the present invention.

FIG. 5 is a conceptual illustration showing how image spatialresolutions are adjusted for an image pair according to the variousembodiments of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention provide systems and methods foradjusting remotely acquired imagery. In particular, the variousembodiments of the present invention provide systems and methods forimproving registration, i.e., alignment, of remotely acquired imagepairs. As previously discussed, using conventional methods for improvingalignment of image pairs is a time-consuming and computation intensiveprocess, especially as the number of data points in such imagesincreases. For example, the typical panchromatic (8000×8000 pixels) andmulti-spectral (2000×2000 pixels for each band) image pair typicallyrequires analysis of over 75 million data points. As a result, whendealing with a large number of images, such analysis methods can beprohibitively long.

However, the amount of computation needed can be reduced under twoconditions: (1) reduced number of data points; and (2) a high degree ofalignment already existing. Therefore, embodiments of the presentinvention are directed at systems and method for generating a mappingfunction based on a comparison of the image pairs converted to differentresolutions. In particular, embodiments of the present invention providesystems and methods for first aligning image pairs downsampled to a lowresolution to generate an initial mapping function (using a low numberof data points). The mapping function can then be refined as image pairsare successively compared at higher resolutions (starting with a highdegree of alignment). Once the mapping function is acceptably refined,the mapping function can be used during an image pair fusion process.

The various embodiments of present invention are specifically embodiedas a method, a data processing system, and a computer program productfor generating mapping functions for image pairs. Accordingly, thepresent invention can take the form as an entirely hardware embodiment,an entirely software embodiment, or any combination thereof. However,the invention is not limited in this regard and can be implemented inmany other forms not described herein. For example, FIG. 1 is aschematic diagram of an embodiment of a computer system 100 forexecuting a set of instructions that, when executed, causes the computersystem 100 to perform one or more of the methodologies and proceduresdescribed herein. In some embodiments, the computer system 100 operatesas a standalone device. In other embodiments, the computer system 100 isconnected (e.g., using a network) to other computing devices. In anetworked deployment, the computer system 100 operates in the capacityof a server or a client developer machine in server-client developernetwork environment, or as a peer machine in a peer-to-peer (ordistributed) network environment.

In the some embodiments, the computer system 100 can comprise varioustypes of computing systems and devices, including a server computer, aclient user computer, a personal computer (PC), a tablet PC, a laptopcomputer, a desktop computer, a control system, a network router, switchor bridge, or any other device capable of executing a set ofinstructions (sequential or otherwise) that specifies actions to betaken by that device. It is to be understood that a device of thepresent disclosure also includes any electronic device that providesvoice, video or data communication. Further, while a single computer isillustrated, the phrase “computer system” shall be understood to includeany collection of computing devices that individually or jointly executea set (or multiple sets) of instructions to perform any one or more ofthe methodologies discussed herein.

The computer system 100 includes a processor 102 (such as a centralprocessing unit (CPU), a graphics processing unit (GPU, or both), a mainmemory 104 and a static memory 106, which communicate with each othervia a bus 108. The computer system 100 further includes a display unit110, such as a video display (e.g., a liquid crystal display or LCD), aflat panel, a solid state display, or a cathode ray tube (CRT)). Thecomputer system also includes an input device 112 (e.g., a keyboard), acursor control device 114 (e.g., a mouse), a disk drive unit 116, asignal generation device 118 (e.g., a speaker or remote control) and anetwork interface device 120.

The disk drive unit 116 includes a computer-readable storage medium 122on which is stored one or more sets of instructions 124 (e.g., softwarecode) configured to implement one or more of the methodologies,procedures, or functions described herein. The instructions 124 reside,completely or at least partially, within the main memory 104, the staticmemory 106, and/or within the processor 102 during execution thereof bythe computer system 100. The main memory 104 and the processor 102 alsocan constitute machine-readable media.

Those skilled in the art will appreciate that the computer systemarchitecture illustrated in FIG. 1 is one possible example of a computersystem. However, the invention is not limited in this regard and anyother suitable computer system architecture can also be used withoutlimitation.

For example, dedicated hardware implementations including, but notlimited to, application-specific integrated circuits, programmable logicarrays, and other hardware devices can likewise be constructed toimplement the methods described herein. Applications that can includethe apparatus and systems of various embodiments broadly include avariety of electronic and computer systems. Some embodiments implementfunctions in two or more specific interconnected hardware modules ordevices with related control and data signals communicated between andthrough the modules, or as portions of an application-specificintegrated circuit. Thus, the exemplary system is applicable tosoftware, firmware, and hardware implementations.

In accordance with various embodiments of the present invention, themethods described below can be stored as software programs in acomputer-readable storage medium and can be configured for running on acomputer processor. Furthermore, software implementations can include,but are not limited to, distributed processing, component/objectdistributed processing, parallel processing, virtual machine processing,which can also be constructed to implement the methods described herein.

Therefore, in some embodiments of the present invention, the presentinvention is embodied as a computer-readable storage medium containinginstructions 124 or that receives and executes instructions 124 from apropagated signal so that a device connected to a network environment126 sends or receive voice and/or video data and that communicate overthe network 126 using the instructions 124. The instructions 124 arefurther transmitted or received over a network 126 via the networkinterface device 120.

While the computer-readable storage medium 122 is shown in an exemplaryembodiment to be a single storage medium, the term “computer-readablestorage medium” should be taken to include a single medium or multiplemedia (e.g., a centralized or distributed database, and/or associatedcaches and servers) that store the one or more sets of instructions. Theterm “computer-readable storage medium” shall also be taken to includeany medium that is capable of storing, encoding or carrying a set ofinstructions for execution by the machine and that cause the machine toperform any one or more of the methodologies of the present disclosure.

The term “computer-readable medium” shall accordingly be taken toinclude, but not be limited to, solid-state memories such as a memorycard or other package that houses one or more read-only (non-volatile)memories, random access memories, or other re-writable (volatile)memories; magneto-optical or optical medium such as a disk or tape; aswell as carrier wave signals such as a signal embodying computerinstructions in a transmission medium; and/or a digital file attachmentto e-mail or other self-contained information archive or set of archivesconsidered to be a distribution medium equivalent to a tangible storagemedium. Accordingly, the disclosure is considered to include any one ormore of a computer-readable medium or a distribution medium, as listedherein and to include recognized equivalents and successor media, inwhich the software implementations herein are stored.

Although the present specification describes components and functionsimplemented in the embodiments with reference to particular standardsand protocols, the disclosure is not limited to such standards andprotocols. Each of the standards for Internet and other packet switchednetwork transmission (e.g., TCP/IP, UDP/IP, HTML, and HTTP) representexamples of the state of the art. Such standards are periodicallysuperseded by faster or more efficient equivalents having essentiallythe same functions. Accordingly, replacement standards and protocolshaving the same functions are considered equivalents.

The present invention will now be described in greater detail inrelation to the flowchart in FIG. 2, illustrating steps in an exemplarymethod 200 for processing remote imagery data based according to thevarious embodiments of the present invention. It should be appreciated,however, that the exemplary process disclosed in FIG. 2 is provided forpurposes of illustration only and that the present invention is notlimited in this regard.

As shown in FIG. 2, the method 200 can start with step 202 and continueon to step 204. In step 204, the remote imagery data is obtained. Asused herein, “remote imagery data” refers to any set of data defining animage pair. That is, the remote imagery data includes image data and anytype of meta-data associated with a first and at least a second image tobe combined. The image data is acquired from any remotely positionedsensor or imaging device. For example, the remote sensor can bepositioned to operate on, by way of example and not limitation, anelevated viewing structure, an aircraft, a spacecraft, or man-madesatellite. That is, the remote data is acquired from any position, fixedor mobile, that is elevated with respect to the imaged location. Theimage data can include light intensity data for an image acquired usingonly a particular range of wavelengths (i.e., a spectral band).Therefore, in the various embodiments of the present invention, theremote imagery data comprises multi-spectral (˜4 bands), hyper-spectral(>100 bands), and/or panchromatic (visible band) image data.

The remote imagery data described herein are further assumed to havecertain other characteristics. During the time between collection of thevarious images, moving objects such as vehicles and ocean waves are notlikely to be registered correctly between the two images, leading toerror in registration and calibration. If the time between theacquisitions of the images is more than a few minutes, the change inposition of the sun will result in significant changes in shadows andvariations in the illumination of the surface based on how well thesurface normals are aligned with the sun. This can result in significantcalibration and registration errors. If days pass between the collectionof the images, there can be significant changes in atmosphericconditions. If months pass, there can be significant changes in thesurface properties due to ice, snow, rain, leaves falling from thetrees, new growth. Therefore, one of ordinary skill in the art willrecognize that better results are obtained in subsequent processes forcombining the images if the different images can be acquired within avery short time frame of each other. Similarly, the different images canalso be acquired from nearly the same position to further reduceregistration errors. Still, it will be understood by those skilled inthe art that the present invention can be utilized in connection withdifferent images that do not satisfy these criteria, possibility withdegraded results. Accordingly, image pairs can be obtained usingdifferently positioned sensors, obtained at different times, or both.However, such image pairs can result in more complex registrationprocesses, including more complex or multiple shifting, scaling, androtation processes. As used herein, a “composite image” refers to anyimage resulting from the combination of any types of image data. Forexample, a composite image is produced from the combination of the imagedata from each spectral band in multi-spectral or hyper-spectralimagery. However, the invention is not limited in this regard and acomposite image can be produced from the fusion of multi-spectral orhyper-spectral image data with panchromatic image data.

For exemplary method 200, the remote imagery data comprises an imagepair including panchromatic and multi-spectral images and associatedmeta-data. By way of example and not limitation, the meta-data includesinformation identifying a date, a time, and the geographic location forthe images. For example, geographic coordinates for the four corners ofa rectangular image can be provided in the meta-data. Other informationcan also be included in the meta-data, including any additionalinformation regarding the sensor or the location being imaged.

Once the image pair is obtained in step 204, an initial mapping functionfor registering, i.e., aligning, the pixels in the image pair is createdin step 206. In general, step 206 involves generating a mathematicalfunction based on determining where each point in the panchromatic imagemaps to coordinates in the multi-spectral image. A number ofconventional methods exist for generating the mapping function.Typically, they involve selecting a number of points in one image,finding where they map to in the other image, and then optimizing thecoefficients of a transform. This is usually a least squares errorsolution that permits one to obtain a set of coefficients that minimizethe squared error of mapping points from one image to another. For bestresults in the fusing process, the panchromatic image is preferablymapped to the multi-spectral image with an accuracy defined by a errordistance which is less than a dimension defined by 0.1 panchromaticpixel. The mapping function created in step 206 determines the mappingof points from the coordinates of one image to the coordinates of theother image. This mapping function can be as simple as a lineartransformation of the form x₁=ax₂+by₂+x₀, or a complex transformationmodeling the geometry of both sensors and the surface imaged. Themapping function can be based on coordinates included within the imagerydata as meta-data. For example, the meta-data can include latitude andlongitude coordinates of the four corners of the acquired image pairsand the initial mapping function can be based on these coordinates.

Once the initial mapping function is created in step 206, themulti-spectral image data is combined to generate an approximatedpanchromatic image in step 208. In particular, the approximatedpanchromatic image is generated from multi-spectral image datacalibrated to accurately correspond to the radiance values of pixels ofthe panchromatic image. Therefore, prior to generating the approximatedpanchromatic image, the spectral weights of the radiance values for thespectral bands comprising the multi-spectral image need to bedetermined. As used herein, the term “radiance value” generally refersto a digital value assigned to a pixel which is intended to representthe intensity of light energy received by a sensor at the locationdefined by that pixel. In this regard, it should be understood thatthese radiance values may be scaled differently in two differentsensors. Accordingly, it will be appreciated that the radiance valuesfrom the two different sensors must somehow be adjusted or scaled byusing suitable weighting factors before the radiance values from the twodifferent sensors can be combined together in a meaningful way. Thisprocess is referred to as calibration.

It must be understood that the complete multi-spectral image of aparticular scene is actually comprised of several optical or spectralimage bands. In each of these bands the sensor is responsive to a verylimited range of optical wavelengths. This concept is illustrated inFIG. 3 which shows curves 301, 302, 303, 304 which represent a sensor'sresponse to four different bands of optical wavelengths. The sensoressentially creates one image for each optical band represented by theresponse curves 301, 302, 303, 304. In this example, a singlemulti-spectral image would be comprised of images obtained by the sensorusing these four spectral bands. Those skilled in the art willappreciate that different sensor systems can have more or fewer opticalbands. In contrast, the panchromatic image is a single image which isobtained by a sensor that is responsive to a much wider range of opticalwavelengths. In FIG. 3, the response of the panchromatic sensor isillustrated by curve 300.

In FIG. 3, it can be seen that the response curves 301, 302, 303, 304 ofthe sensor for the various multi-spectral bands can be very different ascompared to the response curve 300 of the panchromatic sensor for thesame wavelengths. These differences in the responsiveness of the sensorto the various optical bands will result in scaling differences asbetween the radiance values associated with each pixel for themulti-spectral image as compared to the panchromatic image. Therefore acalibration function is needed to scale the radiance values for eachpixel as measured by the multi-spectral sensor to correspond to thescaling of radiance values resulting from the panchromatic sensor. Forexample, consider the spectral response represented by curves 300 and301 at 0.5 μm. The curve 301 has a spectral response of approximately1.0 whereas the spectral response of the panchromatic sensor shows anaverage spectral response in the range of about 0.35. Ignoring for themoment the response of curve 302 in the wavelength range defined bycurve 301, the radiance values for pixels in a multi-spectral imageusing a sensor having the characteristics of response curve 301 wouldlikely need to be scaled by a weighting value of about 0.35 in order forsuch radiance values to be properly calibrated to those values measuredby a sensor having the response indicated by curve 300. In general,proper spectral calibration would require that the pixel radiance valuesassociated with each spectral band in FIG. 3 would need to be addedtogether to obtain a total radiance value that is properly scaled to theradiance values obtained using a sensor having the response defined bycurve 300. This process is illustrated in FIG. 4.

Mathematically, the foregoing process can be expressed as follows inequation (1):

$\begin{matrix}{{P_{MSI}\left( {i,j} \right)} = {{\sum\limits_{\lambda}{W_{\lambda}{M_{\lambda}\left( {i,j} \right)}}} + P_{0}}} & (1)\end{matrix}$

Where:

P_(MSI)(i,j) is the approximated panchromatic radiance of eachdown-sampled pixel;W_(λ) are the spectral weights for each of the spectral bands, λ;M_(λ)(i,j) is the radiance value for each pixel for each spectral bandcomprising the multi-spectral image; andP₀ is a constant offset value.

Once the spectral weights are calibrated, the approximated panchromaticimage is generated in step 208. The approximated image, being derivedfrom a combination of multi-spectral images, will still only have thesame spatial resolution as the multi-spectral images. However, whencompared to the acquired panchromatic image, the differences in radiancevalues between the acquired and approximated panchromatic images shouldonly be minor.

Notably, even if a mapping function is provided for mappingmulti-spectral images to panchromatic images, any differences in spatialresolution and sensor position between the two images always results insome degree of misalignment and therefore problems with registration.Typically, registration problems comprise high resolution pixels fromthe panchromatic image which do not sufficiently overlap pixels in thecorresponding low resolution multi-spectral image, even though bothpixels are imaging the same location. This poor registration can thenresult in improper combination of the image, which causes blurring orother degradation of the composite image being formed. As a result, whena composite image is required at a high resolution, a substantial amountof computation time is required to align the images to properly registerthem.

As previously described, the amount of computation time for properlyaligning images is based on two factors: (1) the resolution of theimages to be combined, and (2) the degree of misalignment between thetwo images. In other words, as either the resolution of the images isreduced or the degree of misalignment is reduced, the amount ofcomputation required for registering the images is also reduced. In thevarious embodiments of the present invention, these factors can be usedadvantageously to register images at different resolutions withoutextensive computations at higher resolutions. In particular, by using acomparison of acquired and approximated panchromatic images at variousresolutions, the mapping function can be iteratively corrected using areduced amount of computation. Thus, in the various embodiments of thepresent invention, the mapping function is coarsely corrected at lowerresolutions, where the amount of computation is limited by the lowernumber of pixels. Subsequently, the mapping function is further adjustedat a higher resolution. However, the amount of computation for thesehigh resolution adjustments is limited since the images have alreadybeen previously aligned at one or more lower resolutions. Therefore, theimages only require only a small degree of correction at higherresolutions. The result is an overall reduced number of computations,even though multiple alignment steps are required.

Accordingly, a first aspect of the present invention provides forgenerating an initial corrected mapping function by aligning the imagesat a low common spatial resolution. That is, the mapping functiongenerated from the meta-data is initially corrected at a resolutionwhere even a poor estimate of the registration is still likely to beonly a fraction of a pixel. For example, as shown in FIG. 5, a lowspatial resolution multi-spectral image 502 and a high spatialresolution image 504 can be converted to low resolution images 506 and508, respectively, having the same resolution. Many downsamplingtechniques are available for converting the images 502 and 504 and canbe used with the various embodiments of the present invention. Forexample, as shown in FIG. 5, pyramid functions 510 and 512 are used forgenerating images 506 and 508, respectively. The same pyramid functions510 and 512 are also used to convert images 502 and 504, respectively,to any other resolutions. In such embodiments, the use of a pyramidfunction ensures that an image is converted to another resolution usingthe same scaling factors and pixel combination techniques. A “pyramidfunction”, as used herein, refers to a function that reduces the numberof pixels in an image by a factor of 2 successively over a number oflayers. At each layer, non-overlapping 2×2 blocks of pixels are averagedtogether to produce a single pixel value in the next layer of the image,as shown in FIG. 5. Thus each layer has ¼ the number of pixels of theprevious layer. Although any resolution (i.e., number of pixels) can beused for the common resolution, reducing the number of pixels below athreshold value can result in loss of too much image information.Accordingly, a minimum number of pixels can be specified for the commonresolution, such as 64 pixels, to ensure that sufficient imageinformation is available for initial registration of the images.

Referring back to FIG. 2, once an approximated panchromatic image isgenerated in step 208, the approximated and acquired panchromatic imagesare first downsampled or converted to a common low resolution below thatof the multi-spectral images in step 210. For example, as shown in FIG.5, images 502 and 504, at resolutions R_(MSI) and R_(Pan), respectively,can be converted to images 506 and 508, respectively, at common lowresolution R_(sub). Subsequently, using the mapping function created instep 206, the images are aligned and compared in step 212 to generate anerror metric measuring a total difference between the radiance valuesfor the two converted images. This initial alignment would be based onthe geographic coordinates in the meta-data, for example. The totaldifference is then measured using an error metric. For example, aroot-mean square (RMS) measurement of the overall difference between theradiance values of the two images can be determined. The error metricwill contain other terms in addition to RMS to ensure a unique anddesirable solution. Specifically we include terms to provide thespatially smoothest mapping between images. Afterwards, in step 214, theerror metric is compared to a threshold value or a determination can bemade of whether the error metric can be reduced further.

In step 216, the mapping function is adjusted to reduce the errormetric. The error metric is reduced by equating the acquired andapproximated panchromatic images at the low spatial resolution. That is,an assumption is made that the radiance of the downsampled panchromaticpixels should be equal to the radiance of the corresponding pixel in thedownsampled approximated panchromatic image. Mathematically, this can beexpressed as follows in equation (2):

P_(MSI)(im,jm)˜P(M2P(im,jm))  (2)

Where, im, im specify the downsampled coordinates in the MSI image,M2P(x,y) specifies the mapping function for mapping the panchromaticimage to the MSI image, and P_(MSI)(x,y) is the acquired panchromaticradiance of each pixel;

Therefore, the mathematical adjustment needed for the mapping functionM2P(x,y) can be derived from equation (2):

$\begin{matrix}{{P_{MSI}\left( {{im},{jm}} \right)} = {P\left( {M\; 2\; {P\left( {{im},{jm}} \right)}} \right)}} \\{{= {{P_{{{ip} + {dy}},{{jp} + {dx}}}0} < {x}}},{{y} < 1}} \\{= {{\left\lbrack {{P_{{ip},{jp}}\left( {1 - {x}} \right)} + {P_{{ip},{{jp} + 1}}{x}}} \right\rbrack \left( {1 - {y}} \right)} +}} \\{{\left\lbrack {{P_{{{ip} + 1},,{jp}}\left( {1 - {x}} \right)} + {P_{{{ip} + 1},,{{jp} + 1}}{x}}} \right\rbrack {y}}}\end{matrix}$

to provide the matrix for determining the adjustment in the mappingfunction as follows in equation (3):

$\begin{matrix}{{{{{\nabla_{x}P}\mspace{14mu} {\nabla_{x}P}}} \times {\begin{matrix}{\delta \; x} \\{\delta \; y}\end{matrix}}} = {{P_{MSI}\left( {{im},{jm}} \right)} - {P\left( {M\; 2\; {P\left( {{im},{jm}} \right)}} \right)}}} & (3)\end{matrix}$

However, using only equation (3) to adjust the mapping function onlyaccounts for the differences in the two images and is typicallyinsufficient to provide a unique solution for the necessary adjustment.Therefore, in step 216, the mapping function also needs to be adjustedaccording to one or more boundary conditions. These boundary conditionsessentially ensure that the location and distribution of radiance valuesin the acquired and approximated panchromatic images matches to somedegree. Accordingly, a unique solution for the adjusted mapping functioncan then be found based on these boundary conditions.

A first and necessary boundary condition is the requirement to generallyimpose smoothness of the mapping between adjacent pixels. Thisrequirement ensures that any features in the images showing a gradual orsmooth change in radiance values in a particular direction ororientation are not significantly altered by the mapping function. Thatis, the first condition requires that a gradient in intensity in aspecific direction needs to be maintained regardless after mappingfunction is updated. In general, the boundary condition can be imposedby using any type of smoothing function. In some embodiments of thepresent invention, this first boundary condition can be imposed by usinga point averaging function to impose smoothness. That is, a position ofa pixel is determined by calculating an average position based on thepositions of existing coordinate pairs and neighbors. Mathematically,this can be expressed as follows in equation (4):

M2P(im,jm)=[ M2P(im+di, jm+dj)+M2P(im−di, jm−dj)]/2  (4)

Therefore, rearranging equation above to minimize difference betweenadjacent pixels, provides an equation (5) for imposing the boundarycondition:

0=½[M2P(im+di, jm+dj)+M2P(im−di, jm−dj)]−M2P(im,jm)  (5)

Therefore:

0=½[M2P(im+di)+M2P(im−di)]−M2P(im)

and

0=½[M2P(jm+dj)+M2P(jm−dj)]−M2P(jm)

impose smoothness in the x and y directions of the remapped image.

A second optional boundary condition can be a requirement that the size,shape, and orientation of objects be preserved after remapping using themapping function. That is, the size, shape, or orientation of an objectin a remapped image should essentially not be distorted after applyingthe mapping function. In general, the boundary condition can be imposedby using any type of rigid object translation function. In someembodiments, this second boundary condition can be imposed by providinga rigid object translation in which an affine transformation is used topreserve the features of objects in the in remapped image.Mathematically, this can be expressed as follows in equation (6):

M2P(im,jm)≈M2P(im+di, jm+dj)−A(di, dj)  (6)

where di,dj˜0,+/−1 are the horizontal and vertical displacements toneighboring pixels and A is derived from an affine transformapproximation of the mapping to provide equation (7) as follows:

(xp, yp)=M2P(im+di, jm+dj)≈A(di, dj)+(xp₀, yp₀)  (7)

Therefore, rearranging equation (7) as shown below in equation (8)provides:

0=(xp, yp)−A(di, dj)−(xp₀, yp₀)  (8)

and more particularly:

0=xp−A(di, dj)−xp₀ and 0=yp−A(di, dj)−yp₀

Which impose rigid object translation in the remapped image in the x andy directions.

As a result of these imposed boundary conditions, the mapping functionneeds to be able to provide handle discontinuities when minimizing thedifference between the acquired and approximated panchromatic images.Accordingly, discontinuities can be handled by applying weights to theboundary conditions based on the similarity of adjacent intensities.Mathematically this can be expressed as follows in equation (9):

$\begin{matrix}{W_{i,j} = {\lambda \left\{ {ɛ + ^{{- {{{M{({{im},{jm}})}} - {M{({{{im} + {di}},{{jm} + {di}}})}}}}^{2}}/\sigma^{2}}} \right\}}} & (9)\end{matrix}$

Where λ˜σ and σ is a LaGrange multiplier expressed asσ=RMS(P(i,j)−P(i,j+1)). These weights are computed at each pixel (i, j)based on the difference of the pixel radiance and the radiance of aneighboring pixel or average radiance of a pair of pixels.

Accordingly, based on the boundary conditions and the weights, thematrix equation in equation (3) can be modified to include the boundaryand discontinuity conditions imposed by equations (5), (8), and (9). Inparticular, equation (3) can be rewritten as a linear system ofequations to provide the matrix equation shown below in equation (10):

$\begin{matrix}{{{\begin{matrix}{\nabla_{x}P} & {\nabla_{y}P} \\W_{0} & 0 \\0 & W_{0} \\\vdots & \vdots \\W_{135} & 0 \\0 & W_{135} \\W_{1} & 0 \\0 & W_{1} \\\vdots & \vdots \\W_{8} & 0 \\0 & W_{8}\end{matrix}}{\begin{matrix}{\delta \; x} \\{\delta \; y}\end{matrix}}} = {\begin{matrix}{{P_{MSI}\left( {i,j} \right)} - {P\left( {M\; 2\; {P\left( {i,j} \right)}} \right.}} \\{W_{0}\left\lbrack {{\left( {{x\left( {i,{j + 1}} \right)} + {x\left( {i,{j - 1}} \right)}} \right)/2} - {x\left( {i,j} \right)}} \right\rbrack} \\{W_{0}\left\lbrack {{\left( {{y\left( {i,{j + 1}} \right)} + {y\left( {i,{j - 1}} \right)}} \right)/2} - {y\left( {i,j} \right)}} \right\rbrack} \\\vdots \\{W_{135}\left\lbrack {{\left( {{x\left( {{i - 1},{j + 1}} \right)} + {x\left( {{i + 1},{j - 1}} \right)}} \right)/2} - {x\left( {i,j} \right)}} \right\rbrack} \\{W_{135}\left\lbrack {{\left( {{y\left( {{i - 1},{j + 1}} \right)} + {y\left( {{i + 1},{j - 1}} \right)}} \right)/2} - {y\left( {i,j} \right)}} \right\rbrack} \\{W_{1}\left\lbrack {{x\left( {i,{j + 1}} \right)} - {A_{x}\left( {0,1} \right)} - {x\left( {i,j} \right)}} \right\rbrack} \\{W_{1}\left\lbrack {{y\left( {i,{j + 1}} \right)} - {A_{y}\left( {0,1} \right)} - {y\left( {i,j} \right)}} \right\rbrack} \\\vdots \\{W_{8}\left\lbrack {{x\left( {{i - 1},{j + 1}} \right)} - {A_{x}\left( {{- 1},1} \right)} - {x\left( {i,j} \right)}} \right\rbrack} \\{W_{8}\left\lbrack {{y\left( {{i - 1},{j + 1}} \right)} - {A_{y}\left( {{- 1},1} \right)} - {y\left( {i,j} \right)}} \right\rbrack}\end{matrix}}} & (10)\end{matrix}$

In equation (10), the first row of the matrix equation minimizes the RMSerror, i.e. the difference in mapping between the approximated andacquired panchromatic images, as previous described in equation (3). Thenext group of rows, associated with four weights W₀, W₄₅, W₉₀ and W₁₃₅,impose the first boundary condition to impose smoothness, as describedabove for equation (8). These rows provide a set of linear equations inthe form:

δx−[(x(di,dj)+x(di,−dj)/2−x]=0 and δy−[(y(di,dj)+y(−di,−dj)/2−y]=0

where i=0, 45,90,135, corresponding to the four lines passing through acenter pixel and the pairs of neighboring pixels in a 3×3 pixel box.That is, each line passes through the center pixel and a first andsecond neighboring pixels on opposite sides of the center pixel. Thefour weights are based on the difference in radiance of the center pixeland the average radiance of the two neighboring pixels affected by theseequations.

Similarly, the final group of rows, associated with the eight weights W₁to W₈, imposes the second boundary condition to preserve shape, size,and orientation of objects in the image, as shown above in equation (5).These eight weights determine how much a given pixel is affected by theeight neighboring pixels in the 3×3 pixel box. These rows provide a setof linear equations in the form:

δx−[x(di,dj)−A _(x)(di,dj)−x]=0 and δy−[y(di,dj)−A _(y)(di,dj)−y]=0

where i=1, 2, . . . 8. The eight weights are based on the difference inradiance of the center pixel and the radiance of the single neighboringpixel affected by these equations.

The adjustment process in step 216 can continue by solving the matrixequation (4) to determine the changes in the mapping function. Thesolution to the equation can be found by using conventional techniques,such as the well known least-squares method:

[A _(t) A]x=A _(t) b

Where multiplying from the left by the transpose of the matrix resultsin a symmetric matrix equation. There are many well know methods forefficiently solving matrix equations of this form.

Once the necessary adjustment for the mapping function is determined instep 216, the downsampled acquired and approximated panchromatic imagesare repeatedly compared in step 212 and the mapping function isrepeatedly adjusted in step 216 until an acceptable solution has beenfound in step 214. However, the resulting mapping function generallywill still require additional adjustment in order to be used forregistering images at higher resolution. That is, even though thedownsampled pixels are aligned, at a higher resolution the pixels may beslightly misaligned. Without correction, at subsequently higherresolutions, the mapping function may not be able provide sufficientlyaccurate registration to align the images. Accordingly, a second aspectof the present invention provides for correcting these smallmisalignments at subsequently higher resolutions. In particular, thevarious embodiments of the present invention provide for essentiallydetermining the amount of adjustment needed at a higher resolution for amapping function corrected at a lower resolution. Therefore, at eachsubsequent higher resolution, the mapping function is further correctedto account for the amount of misalignment resulting as the number ofpixels in the images are increased. However, even though furtheradjustment is needed, computations are limited due to the high degree ofalignment from previous adjustments of the mapping function.

Therefore, once the adjusted mapping function in step 216 results in adownsampled image pair having an acceptable error metric or an errormetric that has been found to be minimized in step 214, the method 200continues to step 218 to perform further adjustments at subsequentlyhigher resolutions. Typically, the purpose for providing an accuratemapping function is to be able to generate a composite image having ahigh spatial resolution and minimal blurring of details. For example, anaccurate mapping function can be needed to generate a fused image havinga high spatial resolution from the combination of a high spatialresolution panchromatic image and an upsampled low resolutionmulti-spectral image. Accordingly, the registration of the upsampled lowresolution images at the high spatial resolution can be critical.

Unfortunately, several difficulties exist for the registration ofupsampled low resolution images with high resolution images. First,there is the basic problem of initial alignment of the images, aspreviously discussed. However, as previously discussed, the variousembodiments of the present invention can provide for basic alignment andregistration of the images by adjusting of the mapping function based ona comparison of radiances of acquired and approximated panchromaticimages. Despite this alignment, other problems in alignment can stillpersist due to the upsampling process. In particular, the upsamplingprocess introduces artifacts into the upsampled images that cannot becorrectly mapped. That is, because the radiances of pixels need to beestimated during the upsampling process, the intensity and distributionof these estimated pixels may not coincide with the intensity of pixelsin the high resolution image. As a result, the composite image can haveone or more portions in which features are blurred due to incorrectmapping of the estimated pixels in the upsampled image to pixels in thehigh resolution pixels.

Consequently, another aspect of the present invention provides foriteratively correcting the mapping function at higher resolutions, butaccounting for artifacts during the upsampling process. In particular,embodiments of the present invention provide for correcting the mappingfunction based not only on a comparison of acquired and approximatedpanchromatic images at subsequently higher resolutions, but also byensuring that the comparison is made using a corrected approximatedpanchromatic image. That is, an approximated panchromatic image that hasbeen corrected for any artifacts being introduced by the upsamplingprocess.

Accordingly, once an initial mapping function is found at step 214, themapping function is adapted for mapping images at a high common spatialresolution higher than that of the low resolution images in step 218.The mapping function, like the images can be adapted using the samescaling function as the image being mapped. For example in the case ofimage 502 in FIG. 5, the mapping function can be adapted using pyramidfunction 510 to a high common spatial resolution comprising a firstintermediate resolution R_(super) _(—) ₁. The factor of two inresolution between layers of the pyramid is not only a computationalconvenient choice. Increasing by a larger factor, such as 3 or 4, wouldreduce the number of layers in the pyramid but would increase the timeto process each layer resulting in slower processing time for the entirepyramid. Furthermore, an increase by only a factor of two in resolutionensures that registration at a given layer is immediately accurate to afraction of a pixel at the next higher layer. Maintaining sub-pixelaccuracy is critical to the bilinear interpolation approach.

However, directly upscaling the multi-spectral image can result inartifacts being introduced into final combined image. These artifactsare due to the upsampled multi-spectral image having many pixelsestimated. For example, in a typical image pair, a panchromatic imagecan have a resolution of 8000 by 8000 pixels, as compared to only2000×2000 pixels for a typical multi-spectral image. Accordingly,upsampling a mutli-spectral image directly to the panchromaticresolution results an additional 60 million pixels being estimated.Consequently, adjustment of the mapping function using such upsampledimages can result in incorrect adjustments due to the mapping functionbeing adjusted for artifacts in the images, rather than actual pixels inthe image.

In the various embodiments of the present invention, the mappingfunction is therefore further adjusted based on downsampled panchromaticimages. This ensures that the mapping function is based on actual pixelsand not based on artifacts. In particular, following step 218, thepanchromatic image is downsampled to the higher common resolution, aresolution above that of the MSI, in step 222. Afterwards, in step 224,the mapping function is used to remap the downsampled panchromatic imageto generate an approximated panchromatic image in the multi-spectralspatial geometry. However, this remapped image will not correlateexactly to an upsampled version of the multi-spectral image andtherefore will not include the differences in the multi-spectral imageneeded for adjusting the mapping function. Therefore, in step 226, ascaling function needs to be generated to adjust the remapped imageprior to reflect the differences between the remapped panchromatic imageand the approximated panchromatic image. That is, a mathematicalfunction that transforms the remapped panchromatic image back into theoriginal image. For example, in some embodiments, the scaling functioncan simply be a scaling factor calculated for each of the pixels:

S _(i,j) =P _(MSI)(i,j)/P _(MSI) _(—) _(Remapped)(i,j)  (11)

Afterwards, the scaling function is upsampled to the high common spatialresolution and used to correct the remapped panchromatic image in step228.

Once the remapped panchromatic image has been properly corrected at thehigh common spatial resolution in step 228 using the scaling function,the mapping function can then be properly adjusted in step 232. In step232, the corrected remapped and the originally downsampled panchromaticimages are registered using the upsampled mapping function generated instep 218. The upsampled mapping image is thus used as initial estimateof the mapping at this higher resolution and is adjusted based on acomparison of the remapped and downsampled panchromatic images. That is,these images are compared to determine an error metric for the imagepair, as previously discussed in step 212. If the error metric can stillbe minimized or is not yet above a threshold value in step 234, themapping function can be adjust in step 236, as previously described instep 216. Afterwards, steps 232-236 are repeated until an error metricthreshold is exceeded or the error metric has been found to be minimizedin step 234.

Once the adjustments at the high common spatial resolution are completedand if the high common spatial resolution is less than that of theacquired panchromatic image resolution in step 238, the mapping functionis adapted to a yet higher resolution in step 240. For example, themapping function can be adapted to a second high common spatialresolution, such as R_(super) _(—) ₂. In the various embodiments, aspreviously described, the use of a pyramid function to upsample imagesand functions ensures that all upsampling or downsampling of images isconsistent. Steps 220-234 are then be repeated to generate an updatedversion of the further upsampled mapping function at the second highcommon spatial resolution. Although FIG. 5 only shows two intermediateresolutions, R_(super) _(—) ₁ and R_(super) _(—) ₂, the invention is notlimited in this regard. One of ordinary skill in the art will recognizeany number of intermediate resolutions can be used. In some embodiments,the mapping function can be upsampled to R_(Pan) without usingintermediate resolutions. If the higher resolution is the panchromaticimage resolution in step 238, the resulting mapping function can then beused in step 240 to register the images. Afterwards, the images can beproperly fused in step 244 and the method can end in step 244.

Upon reviewing the aforementioned embodiments, it would be evident toone of ordinary skill in the art that the embodiments can be modified,reduced, or enhanced without departing from the scope and spirit of theclaims described below.

In first possible alternate embodiment, the error metric calculation instep 232 can be used to further reduce the amount of computation. Forexample, the method 200 can be terminated early if the mapping functionprovides sufficiently accurate registration at a lower resolution. Thatis, if the error metric calculated in step 232 exceeds a second higherthreshold value, it can be assumed that the mapping function willalready provide sufficiently accurate registration at higherresolutions. For example, a first threshold value for an RMS errormetric threshold can be 0.9 and the second threshold can be 0.95.Accordingly, if the RMS error metric is between 0.9 and 0.95, the method200 can proceed to the next high common spatial resolution, aspreviously discussed. However, if the RMS error metric exceeds 0.95, itcan be assumed the further adjustment is not necessary and the method200 can proceed with registering and fusing the image pair. Similarly,if it is determined that the error metric cannot be reduced further, itcan be assumed that any further adjustment of the mapping function willnot provide any additional accuracy in mapping the image pair. Forexample, if the error metric calculated in step 232 is at or near aminimum value, it can be assumed further adjustment will not provideincreased accuracy in mapping.

In a second possible alternate embodiment, the number of additionalintermediate resolutions to use for refining the mapping function can bevaried based on the value of the error metric. For example, if the errormetric calculated exceeds a higher threshold value, as described above,it can be assumed that the mapping function has already been adjusted toprovide a high degree of accuracy between the image pair. Accordingly,in such embodiments, upon exceeding this higher threshold value, thenext high common spatial resolution can be a higher resolution thannormally would be selected. For example, if a sufficiently high RMSvalue is found at R_(super) _(—) ₁, then rather than increasing theresolution in the next iteration to R_(super) _(—) ₂, the resolution caninstead be increased to R_(Pan) or some other high common spatialresolution between R_(Pan) and R_(super) _(—) ₂.

The illustrations of embodiments described herein are intended toprovide a general understanding of the structure of various embodiments,and they are not intended to serve as a complete description of all theelements and features of apparatus and systems that might make use ofthe structures described herein. Many other embodiments will be apparentto those of skill in the art upon reviewing the above description. Otherembodiments can be utilized and derived therefrom, such that structuraland logical substitutions and changes can be made without departing fromthe scope of this disclosure. Figures are also merely representationaland can not be drawn to scale. Certain proportions thereof may beexaggerated, while others may be minimized. Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense.

Such embodiments of the inventive subject matter can be referred toherein, individually and/or collectively, by the term “invention” merelyfor convenience and without intending to voluntarily limit the scope ofthis application to any single invention or inventive concept if morethan one is in fact disclosed. Thus, although specific embodiments havebeen illustrated and described herein, it should be appreciated that anyarrangement calculated to achieve the same purpose can be substitutedfor the specific embodiments shown. This disclosure is intended to coverany and all adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, will be apparent to those of skill in theart upon reviewing the above description.

1. A method for processing remotely acquired imagery, comprising:obtaining imagery data defining a first image of a panchromatic imagetype, said first image having a first spatial resolution; obtainingimagery data defining a second image of a multi-spectral image type,said second image having a second spatial resolution lower than saidfirst spatial resolution; obtaining a mapping function specifying aposition of pixels in said first image with respect to pixels in saidsecond image; adapting said mapping function to a high common spatialresolution higher than said second spatial resolution; generating athird set of imagery data defining a third image of a panchromatic typebased on said first set of imagery data and said adapted mappingfunction and having said high common spatial resolution; and adjustingsaid mapping function based on a first difference between said first andsaid third images at said high common spatial resolution.
 2. The methodof claim 1, wherein the step of adjusting further comprises: minimizingsaid first difference based on at least one condition.
 3. The method ofclaim 1, wherein the step of obtaining further comprises: creating saidmapping function based on said imagery data; generating a fourth set ofimagery data defining a fourth image of a panchromatic type based onsaid second set of imagery data and having said second spatialresolution; downsampling said first and fourth images to a low commonspatial resolution lower than said second spatial resolution; andmodifying said created mapping function based on a second differencebetween said first downsampled and said fourth downsampled images. 4.The method of claim 3, wherein the step of modifying comprises:minimizing said second difference based on at least one condition. 5.The method of claim 3, wherein the step of adjusting further comprises:downsampling said generated third image to said second spatialresolution; correcting said generated third image based on a comparisonof said downsampled third image and said fourth image; and calculatingsaid first difference using said corrected third image.
 6. The method ofclaim 5, the step of correcting further comprising: selecting a scalingfunction to transform said downsampled third image to said generatedfourth image; and applying said scaling function to said generated thirdimage to generate said corrected third image.
 7. The method of claim 1,the method further comprising: if said higher spatial resolution is lessthan said first spatial resolution, repeating said adapting and saidadjusting steps.
 8. A system for processing remotely acquired imagerydata, comprising: a mass storage for receiving imagery data comprisingimagery data defining a first image of a panchromatic image type havinga first spatial resolution and a second image of a multi-spectral imagetype, said second image having a second spatial resolution lower thansaid first spatial resolution; and a processing element configured for:obtaining a mapping function specifying a position of pixels in saidfirst image with respect to pixels in said second image, adapting saidmapping function to a high common spatial resolution higher than saidsecond spatial resolution, generating a third set of imagery datadefining a third image of a panchromatic type based on said first set ofimagery data and said adapted mapping function and having said secondspatial resolution, and adjusting said mapping function based on a firstdifference between said first and said third images at said high commonspatial resolution.
 9. The system of claim 8, wherein said processingelement is further configured to: minimize said first difference basedon at least one condition during said adjusting.
 10. The system of claim8, wherein said processing element is further configured during saidobtaining to: create said mapping function based on said imagery data,generate a fourth set of imagery data defining a fourth image of apanchromatic type based on said second set of imagery data and havingsaid second spatial resolution, downsample said first and fourth imagesto a low common spatial resolution lower than said second spatialresolution, and modify said created mapping function based on a seconddifference between said first downsampled and said fourth downsampledimages.
 11. The system of claim 10, wherein said processing element isfurther configured during said modifying to: minimize said seconddifference based on at least one condition.
 12. The system of claim 10,wherein said processing element is further configured during saidadjusting to: downsample said generated third image to said secondspatial resolution; correct said generated third image based on acomparison of said downsampled third image and said fourth image; andcalculate said first difference using said corrected third image. 13.The system of claim 12, wherein said processing element is furtherconfigured during said correcting to: select a scaling function totransform said downsampled third image to said generated third image;and apply said scaling function to said generated third image togenerate said corrected third image.
 14. The system of claim 8, whereinsaid processing element is further configured to: if said higher spatialresolution is less than said first spatial resolution, repeat saidadapting and said adjusting steps.
 15. A computer-readable storage,having stored thereon a computer program for processing remotelyacquired imagery, the computer program having a plurality of codesections, the code sections executable by a computer to cause thecomputer to perform the steps of: obtaining imagery data defining afirst image of a panchromatic image type, said first image having afirst spatial resolution; obtaining imagery data defining a second imageof a multi-spectral image type, said second image having a secondspatial resolution lower than said first spatial resolution; obtaining amapping function specifying a position of pixels in said first imagewith respect to pixels in said second image; adapting said mappingfunction to a high common spatial resolution higher than said secondspatial resolution; generating a third set of imagery data defining athird image of a panchromatic type based on said first set of imagerydata and said adapted mapping function and having said high commonspatial resolution; and adjusting said mapping function based on a firstdifference between said first and said third images at said high commonspatial resolution.
 16. The computer-readable storage of claim 15,wherein the step of adjusting further comprises: minimizing said firstdifference based on at least one condition.
 17. The computer-readablestorage of claim 15, wherein the step of obtaining further comprises:creating said mapping function based on said imagery data; generating afourth set of imagery data defining a fourth image of a panchromatictype based on said second set of imagery data and having said secondspatial resolution; downsampling said first and fourth images to a lowcommon spatial resolution lower than said second spatial resolution; andmodifying said created mapping function based on a second differencebetween said first downsampled and said fourth downsampled images. 18.The computer-readable storage of claim 17, wherein the step of modifyingcomprises: minimizing said second difference based on at least onecondition.
 19. The computer-readable storage of claim 15, wherein thestep of adjusting further comprises: downsampling said generated thirdimage to said second spatial resolution; correcting said generated thirdimage based on a comparison of said downsampled third image and saidfourth image; and calculating said first difference using said correctedthird image.
 20. The computer-readable storage of claim 19, the step ofcorrecting further comprising: selecting a scaling function to transformsaid downsampled third image to said generated fourth image; andapplying said scaling function to said generated third image to generatesaid corrected third image.